feat(acp,desktop): resume interrupted turns after app restart - #2177
feat(acp,desktop): resume interrupted turns after app restart#2177wpfleger96 wants to merge 1 commit into
Conversation
8d72643 to
a17863c
Compare
|
Thanks for the substantial work on recovery and persistence here. I reviewed the current head ( 1. Membership removal can re-persist an in-flight batch after invalidationIn
If the harness crashes before that prompt completes, a later boot after the agent is re-added can recover work that membership removal intended to purge. Please make invalidation win over the dirty sync—for example, consume/sync the dirty queue state first and perform channel invalidation as the final persistence operation, or introduce a channel-removal transition that excludes/removes that channel's in-flight recovery triggers. Please also add a regression test that dispatches a batch, removes membership while the batch is in flight, immediately reloads the ledger, and verifies that the channel is absent. 2. Replacing an existing ledger is not portable to Windows
std::fs::rename(&tmp_path, path)On Windows, Please use a cross-platform atomic-replacement implementation. Deleting the destination before renaming would not be sufficient because it creates a crash window with no ledger. Please add Windows-executed coverage that persists snapshot A, persists a changed snapshot B to the same path, reloads it, and observes B. The current Windows CI lane compiles The delta since the originally reviewed head |
82520f6 to
9ba32db
Compare
5bae077 to
9185a75
Compare
Managed agents lost any in-flight or queued turn when the app restarted: the event queue lived only in memory, so a mention being worked on at shutdown was silently dropped and never answered. Adds `ledger.rs`, a durable pending-turn mirror written after every queue mutation, and a boot-recovery pipeline in `lib.rs`: staged load, membership gate, TTL expiry, chunked REST fetch with per-event signature/id/channel reconciliation, ordered queue import, unresolved ordering barrier, then one atomic commit. Recovered turns re-run whole (there is no mid-turn checkpoint) and carry a `[restart recovery]` marker so the agent re-reads the thread before redoing work. Exactly-once is enforced on two fronts: a suppression set keyed by event id prevents the boot fetch and live WebSocket delivery from both admitting the same event, and events whose REST fetch failed stay unresolved behind an ordering barrier with a real deadline — a dedicated timer arm in the main `select!` loop fires it even on a completely quiet harness, so the timeout is a bound rather than a hope. `queue.rs` gains payload-owning `complete_batch`, shared `enforce_cap`, `import_recovered`/`admit_recovered`, barrier registration and `dirty_channels` tracking; shutdown-drain now classifies results through `complete_batch` plus a ledger sync at both grace-period sites so a turn completed during drain is not resurrected on the next boot. Off-by-default paths are opt-out via `--resume-on-restart`/`--resume-ttl-secs`. Desktop: adds per-agent resume-on-restart toggle (Advanced settings), makes the toggle authoritative over env precedence (reserved key), and guarantees the ledger file is deleted on every boot where resume is off. Routing fix: handleEditAgent now opens the instance editor for started agents, definition editor for unstarted ones. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
9185a75 to
9c5efca
Compare
|
closing for now after discussing with the team |
Summary
Managed agents lose in-flight work when the app restarts mid-turn. This adds a durable pending-turn ledger to the
buzz-acpharness so an interrupted turn is replayed on the next boot, plus a per-agent Desktop toggle so the behavior can be turned off for a given agent.Harness (
crates/buzz-acp)ledger.rsmirrors the in-memoryEventQueueto disk after every queue mutation, persisting through a temp file plus rename.lib.rs: staged ledger load → membership gate → TTL expiry → chunked REST fetch with per-event reconciliation (signature/id/channel verification) → ordered queue import → unresolved ordering-barrier registration → one atomic commit.tokio::time::Sleeparm in the mainselect!loop fires at the earliest unresolved barrier deadline, so the barrier timeout is a real bound even on a completely quiet harness.complete_batch+ ledger sync at both grace-period drain sites; the join-error arm applies a queue-only panic disposition.queue.rsgainscomplete_batch(atomic payload-owning completion), sharedenforce_cap,import_recovered/admit_recovered,set_unresolved_barrier/next_unresolved_barrier_deadline, anddirty_channelstracking.drain_channeldrops the in-flight batch's recovery mirror alongside the queued events, so the trailing ledger sync cannot re-persist a triggerLedger::invalidate_channeljust purged.in_flight_channels/in_flight_deadlinesstay preserved so the live task still completes or expires.config.rsgains--resume-on-restart/--resume-ttl-secs, withBUZZ_ACP_RESUME_ON_RESTARTas the env equivalent. Resume is on by default.format_promptemits a recovery header and per-event[restart recovery]markers for recovered batches.windows-rustCI job runscargo test -p buzz-acp ledger::, giving the temp-file-plus-rename persist path Windows-native coverage instead of relying on POSIX rename semantics.resume_on_restartis false, boot best-effort deletes the ledger file so toggling off truly drops pending work — a future re-enable starts clean.Desktop toggle
ManagedAgentRecord.resume_on_restartdefaults totruevia serde, so existing stores deserialize unchanged and no migration is needed.BUZZ_ACP_RESUME_ON_RESTARTis reserved (strips from user env-var editor and on-disk records) and emitted unconditionally from spawn so the UI toggle is the sole control plane in both directions — no ambient parent-process leak when ON, no user env override when OFF.spawn_hash, so flipping it raises the restart-required badge rather than leaving a stale process running the old setting.set_managed_agent_resume_on_restartcommand in the same shape as the auto-restart toggle.Dialog routing fix
The agent-profile pencil path (
handleEditAgent) previously short-circuited to the definition dialog for any definition-backed agent, making all instance-level settings unreachable from the UI for agents created via the create flow. The pencil now unconditionally opens the instance-edit dialog, which already handles linked agents (inherited values shown, "edit linked definition" escape hatch wired). The separatehandleEditPersonacallback retains the definition-edit surface for surfaces that explicitly target it.Comment blocks in the touched desktop files are condensed so each file stays within the differential file-size ratchet.
Deferred follow-ups
These were raised as MINOR findings and are non-blocking per reviewer sign-off. Each has a clear fix shape and bounded scope.
--channelsare passed,boot_recoveronly attempts recovery for channels whose membership the harness fetched at startup. A channel added to the CLI args but not yet joined (e.g. a new invite) may have ledger records that silently skip recovery. Fix shape: intersect the staged ledger's channel set with the--channelssubscription set rather than the membership set. Does not affect the Desktop path, which always passes the full subscribed channel list.unresolvedset indefinitely and the ordering barrier fires on every restart. Fix shape: track per-record fetch-attempt count; after a configurable cap (e.g. 3), classify the record as permanently unresolvable and drop it from the ledger.